Skip to content

Add Azure Virtual Machines operators, sensor, and trigger (#49796)#62391

Merged
potiuk merged 1 commit intoapache:mainfrom
nailo2c:feat-49796-azure_virtual_machines_operator
Mar 11, 2026
Merged

Add Azure Virtual Machines operators, sensor, and trigger (#49796)#62391
potiuk merged 1 commit intoapache:mainfrom
nailo2c:feat-49796-azure_virtual_machines_operator

Conversation

@nailo2c
Copy link
Contributor

@nailo2c nailo2c commented Feb 24, 2026

closes: #49796

Why

This request is from issue #49796. ec2 and gce support these features, so I think it would be great if Azure Virtual Machines also supported them.

How

Implemented hooks, operators, sensors, and triggers following the same pattern as the ec2 provider, with deferrable mode support.

The whole process is shown in sequence diagram below:

sequenceDiagram
    participant DAG as DAG
    participant StartOp as StartOperator
    participant Sensor as StateSensor
    participant Trigger as StateTrigger
    participant Hook as AzureComputeHook
    participant Azure as Azure Compute API

    DAG->>StartOp: execute()
    StartOp->>Hook: start_instance(resource_group, vm_name)
    Hook->>Azure: begin_start(...)
    Hook->>Azure: poller.result() (when wait_for_completion=True)
    StartOp-->>DAG: success

    DAG->>Sensor: execute()
    Sensor->>Hook: poke() -> get_power_state()
    Hook->>Azure: instance_view(...)

    alt state already reached
        Sensor-->>DAG: success
    else state not reached + deferrable=True
        Sensor-->>Trigger: defer(trigger=AzureVirtualMachineStateTrigger)
        loop poll until target state
            Trigger->>Hook: async_get_power_state() [native async]
            Hook->>Azure: instance_view(...)
        end
        Trigger-->>Sensor: TriggerEvent(status=success)
        Sensor-->>DAG: execute_complete()
    end
Loading

What

System Test

  1. I prepared Azure cloud resources and DAGs from my local Airflow environment.
    af_49796_1_before_start

  2. Starting
    af_49796_2_starting

  3. Running
    af_49796_3_running

  4. Restarting
    af_49796_4_restarting

  5. Deallocating
    af_49796_5_deallocating

  6. Stopped
    af_49796_6_stopped

Doc

af_49796_7_doc
Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)
    Claude ( I've thoroughly reviewed all AI-generated code :) )

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Copy link
Contributor

@dabla dabla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice and clean code, great work!

@nailo2c
Copy link
Contributor Author

nailo2c commented Feb 24, 2026

Hi, thanks for the review, I learned a lot from it! 😄
If there's anything else I should improve, pleas let me know.

@dabla
Copy link
Contributor

dabla commented Mar 9, 2026

Hi, thanks for the review, I learned a lot from it! 😄 If there's anything else I should improve, pleas let me know.

Just need to fix the spelling wordlist conflict, then I think we can merge.

fix CI MyPy checks error

Replace AirflowException with standard Python exceptions per contribution guidelines

Add Azure Compute connection fields and UI behavior for authentication

Refactor Azure Virtual Machine operators to use a base class and improve connection handling

Add ComputeManagementClient to spelling wordlist

Re-trigger CI
@nailo2c nailo2c force-pushed the feat-49796-azure_virtual_machines_operator branch from 2b09c95 to cae54e4 Compare March 11, 2026 03:04
@potiuk potiuk merged commit 0b7ebb7 into apache:main Mar 11, 2026
135 checks passed
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
…) (apache#62391)

fix CI MyPy checks error

Replace AirflowException with standard Python exceptions per contribution guidelines

Add Azure Compute connection fields and UI behavior for authentication

Refactor Azure Virtual Machine operators to use a base class and improve connection handling

Add ComputeManagementClient to spelling wordlist

Re-trigger CI
PascalEgn pushed a commit to PascalEgn/airflow that referenced this pull request Mar 12, 2026
…) (apache#62391)

fix CI MyPy checks error

Replace AirflowException with standard Python exceptions per contribution guidelines

Add Azure Compute connection fields and UI behavior for authentication

Refactor Azure Virtual Machine operators to use a base class and improve connection handling

Add ComputeManagementClient to spelling wordlist

Re-trigger CI
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Azure Virtual Machines Operator

4 participants